POMDP Structural Results for Controlled Sensing

Structural results for POMDPs are important since solving POMDPs numerically are typically intractable. Solving a classical POMDP is known to be PSPACE-complete [40]. Moreover, in controlled sensing problems [16], [26], [10], it is often necessary to use POMDPs that are nonlinear in the belief state in order to model the uncertainty in the state estimate. (For example, the variance of the state estimate is a quadratic function of the belief.) In such cases, there is no finite dimensional characterization of the optimal POMDP policy even for a finite horizon. The seminal papers [35], [43], [44] give sufficient conditions on the costs, transition provabilities and observation probabilities so that the value function of a POMDP is monotone with respect to the monotone likelihood ratio (MLR) order (and more generally the multivariate TP2 order). These papers then use this monotone result to show that the optimal policy can be lower bounded by a myopic policy. Our recent works [20], [28] relax the conditions on the transition matrix to construct myopic lower and upper bounds.